Information compression system and information compression method

ABSTRACT

The present disclosure provides an information compression system that is capable of achieving higher compression efficiency. A data acquisition section acquires data. A generation section (segmentation section and integration section) determines each object depicted by the data and a sense of each object, and according to results of the determination, generates compression target data by converting values of elements in the data to identification information indicating each object and the sense of each object. A data storage section generates compressed data by compressing the compression target data. This makes it possible to convert highly random element values to slightly random identification information and compress the resulting converted information while reducing the amount of information. Consequently, the compression ratio can be increased.

CROSS-REFERENCE TO PRIOR APPLICATION

This application related to and claims the benefit of priority fromJapanese Patent Application No. 2022-028562, filed Feb. 25, 2022, theentire disclosure of which is incorporated herein by reference.

BACKGROUND OF THE INVENTION Field of the Invention

The present disclosure relates to an information compression system andan information compression method.

Description of the Related Art

In recent years, in the field, for example, of social infrastructure andmobility, sensor data such as multidimensional point cloud data andimage data are acquired by using sensors such as light detection andranging (LiDAR) sensors and cameras, and utilized for a variety ofapplications. However, for example, the above-mentioned field encountersa problem that the amount of sensor data becomes huge.

In view of the above circumstances, a technology for compressingmultidimensional data by using a neural network is disclosed inJP-2021-111882-A. This technology is capable of achieving optimalcompression regardless of the dimensionality and format of themultidimensional data.

SUMMARY OF THE INVENTION

However, the multidimensional data is compressed as is by the technologydescribed in JP-2021-111882-A. Therefore, sufficient compressionefficiency may not be achieved in some cases.

It is an object of the present disclosure to provide an informationcompression system and an information compression method that arecapable of achieving higher compression efficiency.

According to an aspect of the present disclosure, there is provided aninformation compression system that compresses data and includes anacquisition section, a generation section, and a compression section.The acquisition section acquires the data. The generation sectiondetermines each object depicted by the data and a sense of each objectand generates compression target data according to results of thedetermination. The compression target data is obtained by convertingvalues of individual elements in the data to identification informationthat indicates each object and the sense of each object. The compressionsection generates compressed data by compressing the compression targetdata.

The present disclosure provides higher compression efficiency.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagram illustrating a physical configuration of aninformation compression system according to a first embodiment of thepresent disclosure;

FIG. 2 is a diagram illustrating a logical configuration of a node;

FIG. 3 is a diagram illustrating an example of sensor data;

FIG. 4 is a diagram illustrating an example of integrated data;

FIG. 5 is a diagram illustrating an example of sensor information;

FIG. 6 is a diagram illustrating an example of chunk information;

FIG. 7 is a diagram illustrating an example of a management table;

FIG. 8 is a diagram illustrating an example of a data usage interface;

FIG. 9 is a diagram illustrating an example of a setting interface;

FIG. 10 is a flowchart illustrating an example of a write process;

FIG. 11 is a flowchart illustrating an example of a read process;

FIG. 12 is a diagram illustrating an example configuration of a datastorage section according to the first embodiment of the presentdisclosure;

FIG. 13 is a diagram illustrating an example configuration of the datastorage section according to a second embodiment of the presentdisclosure;

FIGS. 14A and 14B are diagrams illustrating a conversion processperformed by an identification (ID) conversion compressor;

FIG. 15 is a diagram illustrating an example configuration of the datastorage section according to a third embodiment of the presentdisclosure;

FIG. 16 is a diagram illustrating another example configuration of thedata storage section according to the third embodiment of the presentdisclosure;

FIG. 17 is a diagram illustrating an example configuration of the datastorage section according to a fourth embodiment of the presentdisclosure; and

FIG. 18 is a diagram illustrating an example configuration of the datastorage section according to a fifth embodiment of the presentdisclosure.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

Embodiments of the present disclosure will now be described withreference to the accompanying drawings.

First Embodiment

FIG. 1 is a diagram illustrating a physical configuration of aninformation compression system according to a first embodiment of thepresent disclosure. The information compression system depicted in FIG.1 includes a node 1. The node 1 is communicatively connected to a sensor2 and an input/output device 3. Further, a plurality of nodes 1 may beincluded in the information compression system. In the case where theinformation compression system includes two or more nodes 1, at leastone of the nodes 1 should be connected to the sensor 2 and theinput/output device 3. In the example of FIG. 1 , two nodes 1 arecommunicatively connected to each other through a network 4, with thesensor 2 connected to one of the nodes 1.

The node 1 is a computer system such as a cloud system, an on-premsystem, an edge computing system, or a smartphone or other mobileequipment.

The node 1 includes a main processor 11, a main memory 12, a storage 13,interfaces (I/Fs) 14 and 15, and a sub-processing unit 16. Thesecomponents are interconnected through an internal bus 17.

The main processor 11, which is, for example, a central processing unit(CPU), loads a program (computer program) into the main memory 12 fromthe storage 13 and executes the program to perform various processesaccording to the program. The main memory 12 is a storage device that isused as a work area of the program. The storage 13 is a storage devicefor storing, for example, the program and information used or generatedby the main processor 11 and the sub-processing unit 16. The I/Fs 14 and15 are communication devices for communicatively connecting to anexternal device. In FIG. 1 , the I/F 14 is connected to the sensor 2,and the I/F 15 is connected to a remote node 1 through the network 4.

The sub-processing unit 16, which is, for example, a graphics processingunit (GPU), performs a predetermined process according to the program.The sub-processing unit 16 includes a plurality of cores 16 a and asub-memory 16 b. The cores 16 a are used to perform multiprocessing forsimultaneously performing a plurality of processes. The sub-memory 16 bis used as a work area by the cores 16 a.

The sensor 2, which is a sensing device such as a Lidar or other opticalsensor, an optical camera, or a gravity sensor, transmits detectedsensor data to the node 1. Alternatively, an SD card or other recordingmedium storing sensor data may be used instead of the sensor 2 or inaddition to the sensor 2.

The input/output device 3 includes an input device and an output device.The input device is a keyboard, a touch panel, a pointing device, orother device that receives various kinds of information from a user ofthe information compression system. The output device is a displaydevice, a printer, or other device that outputs various kinds ofinformation to the user. Alternatively, the input/output device 3 maybe, for example, a mobile terminal used by the user.

FIG. 2 is a diagram illustrating a logical configuration of the node 1.As depicted in FIG. 2 , the node 1 includes a data acquisition section101, a segmentation section 102, an integration section 103, a datastorage section 104, a data conversion section 105, and a data usagesection 106. The individual sections 101 to 106 of the node 1 areimplemented when, for example, at least either one of the main processor11 and the sub-processing unit 16 executes the program.

The data acquisition section 101 acquires sensor data from the sensor 2.In the present embodiment, the sensor data includes point cloud data andcolor camera data. The point cloud data is three-dimensional dataacquired by a measurement sensor such as a Lidar or time-of-flight (TOF)sensor. The color camera data is image data acquired by a color camera.The point cloud data is an aggregate of point data pieces indicatingindividual positions of an object surface. Each point data pieceincludes coordinate information indicating the coordinates of theposition of the object surface. The color camera data includes pixelvalues of each pixel that represent color information regarding aplurality of colors (red (R), green (G), and blue (B) in the presentembodiment). Further, in the present embodiment, the point cloud dataand the color camera data are time-series data. More specifically, thecolor camera data is video data.

The data acquisition section 101 may provide the user with a data usageinterface for designating the sensor data to be acquired and acquire thesensor data designated by the data usage interface. Alternatively, forexample, a command or an application programming interface (API) may beused instead of the data usage interface in order to designate thesensor data to be acquired.

The segmentation section 102 determines each object depicted by thecolor camera data acquired by the data acquisition section 101 and thesense of each object, and generates segmentation data according to theresults of the determination. The segmentation data is obtained byconverting the individual pixel values of the color camera data toidentification information that indicates each object and the sense ofeach object. The identification information includes an instance ID anda sense ID. The instance ID is the identification information foridentifying the object. The sense ID is the identification informationfor identifying the sense of the object. It should be noted that thesegmentation data may be obtained by converting at least some of theindividual pixel values to the identification information.

For a segmentation process performed by the segmentation section 102,for example, a segmentation model and a management table are used. Thesegmentation model is a learned model for determining each objectdepicted by the color camera data and the sense of each object. Themanagement table is used to define the instance ID and the sense ID. Thesegmentation section 102 may provide the user with a setting interfacefor defining the settings related to the segmentation process andperform the segmentation process according to the settings definedthrough the setting interface.

The integration section 103 generates integrated data based on sensorinformation regarding the sensor 2 by integrating the point cloud data,which is acquired by the data acquisition section 101, with thesegmentation data, which indicates the result of processing performed bythe segmentation section 102. More specifically, the integrated dataincludes coordinate information and identification information regardingeach position of the object surface depicted by the color camera data.The coordinate information indicates the coordinates of the position,whereas the identification information indicates the object existing atthe position. In the present embodiment, the integrated data serves ascompression target data, which is to be actually compressed. Thesegmentation section 102 and the integration section 103 form ageneration section for generating the compression target data. It shouldbe noted that the integration section 103 uses sensor information 50 toconvert the point cloud data, which is acquired by the data acquisitionsection 101, to a unified coordinate space (coordinate space designatedby the data usage interface 80). Therefore, the data can be handled in aunified manner by the data usage interface 80. Additionally, in a casewhere the same object is imaged by a multi-view sensor, the amount ofpoint cloud data, which becomes redundant, can efficiently be reduced bylater-described quantization and compression processing.

The data storage section 104 functions as a compression section forgenerating compressed data by compressing the integrated data, which isintegrated by the integration section 103, and functions as adecompression section for generating decompressed data by decompressingthe compressed data.

For example, the data storage section 104 compresses the integrated datain units of data blocks called chunks. Further, the data storage section104 may store the compressed data in the node 1, which is a local nodehaving the data storage section 104 that has compressed the data, or maystore the compressed data in a remote node 1 by transferring thecompressed data to the remote node 1 other than the local node.Furthermore, the data storage section 104 reads out and decompresses thecompressed data at a predetermined timing or in response to userinstructions.

The data conversion section 105 converts the decompressed data, which isgenerated by the data storage section 104, to data in a predeterminedformat. For example, the data conversion section 105 converts thedecompressed data to mesh data or to learning data for machine learning.Alternatively, the decompressed data may be used as is without beingconverted.

The data usage section 106 receives converted data, which is convertedby the data conversion section 105, and supplies the converted data asoutput data. For example, the data usage section 106 displays the outputdata on the input/output device 3 or transmits the output data to theremote node 1 for the purpose, for example, of utilizing thedecompressed data in a real-time manner or in a time-series manner. Thedecompressed data may be utilized in the real-time manner, for example,for inference or visualization and may be utilized in the time-seriesmanner, for example, for learning or analysis.

FIG. 3 is a diagram illustrating an example of the sensor data. Sensordata 30 depicted in FIG. 3 includes point cloud data 30 a and colorcamera data 30 b.

The point cloud data 30 a is an aggregate of point data pieces 31, whichis coordinate information indicating positions of an object surface.Each point data piece indicates the coordinates of a position of theobject surface by using a Cartesian coordinate system that is defined bythe x-axis, the y-axis, and the z-axis. The x-, y-, and z-axes may bedefined for each sensor 2 or broadly defined.

The color camera data 30 b indicates pixel values of individual pixelsthat are arrayed in a matrix form in two-dimensional directions, namely,in the horizontal and vertical axis directions. The pixel values containcolor information that includes a plurality of values representing aplurality of different colors (red (R), green (G), and blue (B) in thepresent embodiment). Therefore, the color camera data 30 b may beregarded as three-dimensionally arrayed data ([3] [X] [Y]). In thiscase, [3] indicates the color information, [X] indicates a pixelposition in the horizontal axis direction, and [Y] indicates the pixelposition in the vertical axis direction. For the sake of simplicity,FIG. 3 depicts only the pixel values corresponding to a single color.

In the present embodiment, the point cloud data 30 a and the colorcamera data 30 b are time-series data. The data depicted in FIG. 3 arethe point cloud data 30 a and the color camera data 30 b at a certainpoint of time.

FIG. 4 is a diagram illustrating an example of the integrated data.Integrated data 40 depicted in FIG. 4 includes fields 41 to 43. Thefield 41 stores the coordinate information (point data) that indicates aposition of an object surface. The field 42 stores the sense ID thatidentifies the sense of the object existing at the position indicated bythe coordinate information stored in the field 41. The field 43 storesthe instance ID that identifies the object existing at the positionindicated by the coordinate information stored in the field 41.

FIG. 5 is a diagram illustrating an example of the sensor information.The sensor information 50 depicted in FIG. 5 includes fields 51 to 59.

The field 51 stores a sensor ID that is identification information foridentifying the sensor 2. The field 52 stores the type of the sensor 2that is identified by the sensor ID stored in the field 51. Two types ofsensors 2, namely, “point cloud” and “color camera,” are described inthe present embodiment. “Point cloud” corresponds to the sensor 2 (e.g.,Lidar sensor) that acquires the point cloud data. “Color camera”corresponds to the sensor 2 (e.g., color camera) that acquires the colorcamera data. The field 53 stores a pair ID identifying a pair of sensors2 that acquire the point cloud data and the color camera data forgenerating the integrated data. In the example of FIG. 5 , the sensor IDof the “point cloud data” sensor 2 paired with the “color camera” sensor2 is stored as the pair ID in the field 53 corresponding to the field 52where “color camera” is stored as the type of sensor 2.

The field 54 stores position information that indicates the positionwhere the sensor 2 is disposed. The position information indicates theposition of the sensor 2 by using the Cartesian coordinate system withx-, y-, and z-axes. It should be noted that the coordinate axes (x-, y-,and z-axes) for defining the position of the sensor 2 need not beidentical with the coordinate axes of the point cloud data depicted inFIG. 3 . The field 55 stores orientation information that indicates theorientation of the sensor 2. In the example of FIG. 5 , the orientationinformation is represented by a rotation angle Ψ, an elevation angle θ,and an azimuth angle ϕ. The field 56 stores a scale of the sensor 2. Thefield 57 stores a focal length of the sensor 2. The field 58 stores aresolution of the sensor 2. The field 59 stores an angle of view of thesensor 2.

FIG. 6 is a diagram illustrating an example of chunk informationregarding a chunk that is a data block unit used for compressing theintegrated data. Chunk information 60 depicted in FIG. 6 includes fields61 to 68.

The field 61 stores the sensor ID for identifying the sensor 2 that isused to generate the integrated data to be compressed. The field 62stores an x-direction start position of a chunk in the integrated data.The field 63 stores a y-direction start position of the chunk in theintegrated data. The field 64 stores a z-direction start position of thechunk in the integrated data. The field 65 stores a start time of thechunk in the integrated data. The field 66 stores an end time of thechunk in the integrated data. The width of the chunk in each of the x-,y-, and z-directions is, for example, predesignated separately from thechunk information 60. The field 61 may alternatively store a pluralityof sensor IDs for storing the information acquired from a plurality ofsensors as the information regarding the same chunk.

The field 67 stores a compression state of the chunk. The compressionstate indicates whether the chunk is compressed or not. In the casewhere the chunk is compressed, the compression state additionallyindicates a compression algorithm used for chunk compression. The field68 stores the compressed data that is obtained by compressing the chunk.The compressed data includes, for example, compressed binary data, whichis a compressed chunk main body, reference information indicating amanagement table used for chunk compression, and settings related tonormalization performed at the time of compression. The referenceinformation is, for example, a pointer pointing to the management table.The settings are, for example, the minimum and maximum values regardingeach coordinate axis in a case where normalization is performed by themin-max method.

In the example of FIG. 6 , the chunk is set in terms of position andtime. However, the method of setting the chunk is not limited to themethod depicted in the example of FIG. 6 . For example, the chunk may beset in terms of at least either one of the instance ID and the sense ID.

FIG. 7 is a diagram illustrating an example of the management table. Amanagement table 70 depicted in FIG. 7 includes a sense management table70 a and an instance management table 70 b.

The sense management table 70 a includes fields 71 and 72. The field 71stores the sense ID. The field 72 stores sense information thatindicates the sense identified by the sense ID. In the presentembodiment, the sense information indicates the type of object, such as“person” or “desk,” as the sense.

The instance management table 70 b includes fields 73 and 74. The field73 stores the instance ID. The field 74 stores a broad area ID thatbroadly identifies the object identified by the instance ID. Theinstance ID is identification information for identifying an objectaccording to a single piece of integrated data (or a single targetspace), whereas the broad area ID is identification information foridentifying a target object according to all pieces of integrated data.

FIG. 8 is a diagram illustrating an example of the data usage interfaceused for reading out the output data through the data usage section 106.The data usage interface 80 depicted in FIG. 8 includes designationfields 81 to 87.

The designation field 81 is used for designating the sensor ID thatidentifies the sensor 2 associated with the output data to be read out.The designation field 82 is used for designating a start position in thespace of the sensor data to be acquired. The designation field 83 isused for designating an end position in the space of the output data tobe read out. In the example of FIG. 8 , the x-, y-, and z-coordinates ofthe start position and the end position are designated. The designationfield 84 is used for designating a start time of the output data to beread out. The designation field 85 is used for designating an end timeof the output data to be read out. The designation field 86 is used fordesignating the sense ID that indicates the sense of the output data tobe read out. The designation field 87 is used for designating theinstance ID of the output data to be read out. Designating the sense IDand the instance ID results in acquiring only the output datacorresponding to the designated IDs.

The designation fields 81 to 87 may be set to “Any” for designating all.Further, “real time” may be designated as the start time and the endtime. When “real time” is designated in such a manner, the output datacorresponding to the sensor data at the present time, which is acquiredby the data acquisition section 101, is read out as a stream in realtime through the data usage section 106.

FIG. 9 is a diagram illustrating an example of the setting interface fordefining the settings related to the segmentation process performed bythe segmentation section 102. A setting interface 90 depicted in FIG. 9includes selection fields 91 to 93 and setting buttons 94 and 95.

The selection field 91 is used for designating a storage location of thesegmentation model to be used for the segmentation process. Theselection field 92 is used for designating a storage location of themanagement table to be used for the segmentation process. The selectionfield 93 is used for describing the conversion of the color camera datato the segmentation data in the segmentation process and specifyingwhether data acquisition is necessary. More specifically, the selectionfield 93 is used for defining the sense of the object targeted for dataacquisition and specifying whether the pixel values of the relevantpixel are to be converted to the identification information or leftunchanged and used as the color information. For example, when the sense“desk” is set as unnecessary, a data portion determined as “desk” in theresult of segmentation is deleted and will not be stored. This functionsuppresses the acquisition of unnecessary data and saves the storagecapacity.

The setting button 94 is used for setting the segmentation model and themanagement table. When pressed, the setting button 94 sets thesegmentation model and the management table that are stored in thestorage locations designated in the selection fields 91 and 92. Thesetting button 95 is used for setting the conversion description. Whenpressed, the setting button 95 sets the conversion description.

Using the setting interface makes it possible to delete the colorinformation with respect to a “person” or other object having a specificsense, and substitute the sense ID and the instance ID for the deletedcolor information. This provides privacy protection.

FIG. 10 is a flowchart illustrating an example of a write process thatis performed to compress and store the sensor data.

In the write process, first of all, the data acquisition section 101acquires the sensor data from the sensor 2 (step S101). The acquiredsensor data includes the point cloud data and the color camera data.

The segmentation section 102 analyzes the color camera data in thesensor data, which is acquired by the data acquisition section 101, byusing the segmentation model and the management table set by the settinginterface, and thus determines the object depicted by the color cameradata and the sense of the object. Then, the segmentation section 102acquires the instance ID, which is the identification information foridentifying the determined object, and the sense ID, which is theidentification information for identifying the sense of the object (stepS102).

The segmentation section 102 converts the color camera data to thesegmentation data, according to the conversion description set by thesetting interface 90 (step S103). As a result, filtering is performed insuch a manner that only pixels depicting the object having the sense setby the setting interface 90 remain in the segmentation data as theidentification information or the color information.

The integration section 103 generates the integrated data according tothe sensor information by integrating the point cloud data and thesegmentation data, which correspond to each other (step S104). In a casewhere the point cloud data and the color camera data are acquired hereas the sensor data, the integration section 103 generates the integrateddata by giving the identification information to the coordinate pointsof the point cloud data corresponding to spatial positions of the pixelsin the segmentation data. Further, even in a case where the dataacquisition section 101 does not acquire the point cloud data, there isa way to generate the integrated data. More specifically, theintegration section 103 may acquire the segmentation data from the colorcamera data, calculate a depth map (the distance between the sensor andthe object corresponding to the pixels) by a common method such as thedepth estimation method, and calculate the coordinate points in thespace from the calculated depth map. This enables the integrationsection 103 to acquire information similar to the point cloud data, andsimilarly generate the integrated data having the information regardingthe coordinate points by using the acquired similar information.

The data storage section 104 divides the integrated data, which isgenerated by the integration section 103, into a plurality of chunk datapieces, and generates the chunk information regarding each chunk datapiece (step S105).

The data storage section 104 normalizes each chunk data piece (stepS106). Here, the data storage section 104 normalizes each chunk datapiece by the min-max method.

The data storage section 104 determines at this timing whether or not toperform synchronous compression for compressing the chunk data pieces(step S107). It should be noted that whether or not to performsynchronous compression is, for example, preset.

In the case where synchronous compression is to be performed (“YES” atstep S107), the data storage section 104 performs quantization on eachchunk data piece (step S108). Here, the quantization is, for example, aprocess of dividing the point cloud data expressed by floating pointcoordinates by a value called a quantization width, and converting theresultant to an integer through an operation based, for example, on theROUND function. At the time of normalization, quantization granularitycan be adjusted by using the quantization width. For example, in thecase of the point cloud, duplicated identical coordinate points mayarise after quantization. Therefore, data volume reduction canefficiently be achieved by deleting such duplicated identical coordinatepoints. Here, the sub-processing unit 16 is able to rapidly execute thedeletion of the duplicated identical coordinate points, for example, byusing the UNIQUE function, which eliminates duplicated elements in amachine learning processing system. Further, decreasing the quantizationgranularity causes a decrease in accuracy as well as the amount of data,whereas increasing the quantization granularity causes an increase inthe accuracy as well as the amount of data. That is, the balance betweenthe amount of data and the accuracy of coordinates can be adjusted bythe quantization granularity (quantization width). Moreover, when thesensor data pieces obtained by imaging the same target object from aplurality of points of view are simultaneously processed, moreduplicated elements can efficiently be eliminated by the above-describedquantization and deletion of identical coordinate points. Therefore,total data volume can efficiently be reduced. Subsequently, the datastorage section 104 generates the compressed data, which is obtained bycompressing each quantized chunk data piece, as target data (step S109).Meanwhile, in the case where synchronous compression is not to beperformed (“NO” at step S107), the data storage section 104 skips stepsS108 and S109 and regards the chunk data piece as the target data.

Subsequently, the data storage section 104 determines at this timingwhether or not to perform synchronous transfer for transferring eachtarget data to the remote node (step S110). It should be noted thatwhether or not to perform synchronous transfer is, for example, preset.

In the case where the synchronous transfer is to be performed (“YES” atstep S110), the data storage section 104 transfers the target data tothe remote node 1 (step S111). Subsequently, upon receiving the targetdata, the data storage section 104 in the remote node 1 stores thereceived target data (step S112), and terminates the write process.Meanwhile, in the case where the synchronous transfer is not to beperformed (“NO” at step S110), the data storage section 104 skips stepS111, then stores the target data in the local node 1 (step S112), andterminates the write process.

The individual steps (steps S101 to S112) of the above-described writeprocess may be performed by separate nodes 1. In such a case, a transferprocess of transferring data to a remote node 1 is performed between theindividual steps. Further, each chunk data piece that is notsynchronously compressed may also be compressed at an appropriatetiming. Furthermore, the target data that is not synchronouslytransferred may also be transferred to the remote node 1 at anappropriate timing.

FIG. 11 is a flowchart illustrating an example of a read process that isperformed to decompress and output the compressed data.

In the read process, the data storage section 104 determines a chunkdata piece targeted for decompression as the target chunk data (stepS201). For example, the data usage section 106 provides the user withthe data usage interface and allows the user to designate the chunk datapiece targeted for decompression, whereas the data storage section 104determines the user-designated chunk data piece as the target chunkdata.

The data storage section 104 determines whether or not the target chunkdata is stored in the local node (step S202).

In the case where the target chunk data is stored in the local node(“YES” at step S202), the data storage section 104 reads the targetchunk data (step S203). Meanwhile, in the case where the target chunkdata is not stored in the local node (“NO” at step S202), the datastorage section 104 reads the target chunk data from a remote node 1where the target chunk data is stored (step S204).

Subsequently, the data storage section 104 determines whether or not theread target chunk data is compressed (step S205).

In the case where the target chunk data is compressed (“YES” at stepS205), the data storage section 104 decompresses the target chunk data(step S206). The data storage section 104 performs inverse quantizationon the decompressed target chunk data (step S207), and then performsrenormalization (step S208). Here, the inverse quantization is a processof returning to an original scale value by multiplying the target chunkdata by the quantization width used for compression. In the case wherethe target chunk data is not compressed (“NO” at step S205), the datastorage section 104 skips steps S206 to S208.

Subsequently, the data storage section 104 couples the target chunk datatogether to generate the decompressed data (step S209). It should benoted that the decompressed data is the integrated data in a case wherethe chunk data is lossless compressed.

The data conversion section 105 converts the decompressed data, which isgenerated by the data storage section 104, to data in a predeterminedformat, then outputs the resulting converted data (step S210), andterminates the read process.

The individual steps (steps S201 to S210) of the above-described readprocess may be performed by separate nodes 1. In such a case, a transferprocess of transferring data to a remote node 1 is performed between theindividual steps.

FIG. 12 is a diagram illustrating a configuration of the data storagesection 104 in greater detail. The data storage section 104 includes anormalizer/quantizer 201, a voxelizer 202, an entropy estimator 203, andan entropy encoder 204 as compression processing components, andincludes an entropy decoder 211, an entropy estimator 212, a pointclouder 213, and an inverse quantizer/renormalizer 214 as decompressionprocessing components. The entropy estimators 203 and 212 may have thesame configuration.

In the compression processing, first of all, the normalizer/quantizer201 performs normalization and quantization on the coordinateinformation in the chunk data. The normalization and the quantizationare performed on each of the coordinate axes (x-, y-, and z-axes)defining the three-dimensional space.

Subsequently, the voxelizer 202 generates voxel information byvoxelizing the quantized chunk data, which is obtained by normalizingand quantizing the coordinate information. More specifically, thevoxelizer 202 divides the chunk data into a plurality of voxels, whichare three-dimensional regions having a predetermined volume, and setsthe value of each voxel according to the identification information(sense ID and instance ID) corresponding to the individual coordinatescontained in the voxel. More specifically, the value of each voxel isrepresented by the sense ID and instance ID that correspond to theindividual coordinates contained in the voxel and that are greatest innumber. As a result, the chunk data is converted to the voxelinformation, which is an aggregate of a voxel Ch1 and a voxel Ch2. Thevoxel Ch1 has a value representing the sense ID (S). The voxel Ch2 has avalue representing the instance ID (I).

In a case where the color information, instead of the sense ID and theinstance ID, corresponds to the individual coordinate information, thevoxel information is an aggregate of a voxel Ch3, a voxel Ch4, and avoxel Ch5. The voxel Ch3 has a value representing a red color (R). Thevoxel Ch4 has a value representing a green color (G). The voxel Ch5 hasa value representing a blue color (B). Further, each voxel may berepresented by an octree.

The entropy estimator 203 estimates entropy of the voxel information.Here, the entropy estimator 203 estimates, as the entropy, probabilitydistribution indicating an appearance probability of each symbol that iscapable of representing the value of the voxel information (hereinaftermay be referred to simply as the probability distribution). The entropyestimator 203 is built, for example, by a learned model based on the useof a deep neural network (DNN) such as a multilayer three-dimensionalconvolutional neural network (CNN). The entropy estimator 203 may inputlow-resolution voxel information and estimate the probabilitydistribution of high-resolution voxel information. In this case, thevoxel information having different resolutions may be inputted to theentropy estimator 203 to gradually estimate the probability distributionfor the purpose of providing improved prediction accuracy and variouskinds of resolution decoding (generally called progressive decoding).Further, for the purpose of estimation accuracy improvement, highlysimilar past voxel information or statistically processed voxelinformation (e.g., voxel information processed to determine the median,mean, or variance of a predetermined period) may be inputted when thetime-series data is handled. Further, for the purpose of estimationaccuracy improvement, the probability distribution of a symbol targetedfor estimation may be estimated by using the multilayerthree-dimensional CNN as an autoregressive model and inputting thesymbol value of known voxel information, for example, in the vicinity ofthe estimation target, to the entropy estimator 203. Moreover, increasedefficiency may be achieved, for example, by matching the dataresolutions of a plurality of data inputs to the entropy estimator 203,which relate to the above-mentioned methods, coupling the resulting datapieces having the matched resolution, and inputting the coupled data toa multilayer three-dimensional CNN channel.

Subsequently, the entropy encoder 204 generates the compressed binarydata by encoding the voxel information according to the probabilitydistribution estimated by the entropy estimator 203.

Further, in decompression processing, the entropy decoder 211 generatesthe voxel information by decoding the compressed binary data. Morespecifically, the entropy decoder 211 uses the entropy estimator 212 topredict the probability distribution of a voxel value (symbol), uses thecompressed binary data and the predicted probability distribution todecode the symbol, and finally decodes it as the voxel information. Inorder to obtain the estimation result indicating the same probabilitydistribution as at the time of encoding, the same entropy estimator asthe entropy estimator 203 is used as the entropy estimator 212. Further,the input to the entropy estimator 212 is the same as the input to theentropy estimator 203 at the time of encoding. Moreover, in the casewhere the probability distribution is estimated gradually at differentresolutions at the time of compression or estimated based on theautoregressive model, probability distribution estimation by the entropyestimator 212 and voxel information decoding by the entropy decoder 211are repeated multiple times to perform decoding to generate the finalvoxel information.

The point clouder 213 converts the voxel information, which is generatedby the entropy decoder 211, to the quantized chunk data having thecoordinate information and the identification information. The inversequantizer/renormalizer 214 generates the chunk data, which serves as thedecompressed data, by performing inverse quantization andrenormalization on the coordinate information regarding the quantizedchunk data, which is generated by conversion performed by the pointclouder 213.

As described above, according to the present embodiment, the dataacquisition section 101 acquires the color camera data. The generationsection (segmentation section 102 and integration section 103)determines each object depicted by the color camera data and the senseof each object, and according to the results of the determination,generates the compression target data by converting the pixel values ofthe color camera data to the identification information indicating eachobject and the sense of each object. The data storage section 104generates the compressed data, which is obtained by compressing thecompression target data. This makes it possible to convert highly randompixel values to slightly random identification information and achievecompression while reducing the amount of information. Consequently, thecompression ratio can be increased.

Moreover, in the present embodiment, the data acquisition section 101further acquires the point cloud data including a plurality of pieces ofcoordinate information indicating the individual positions of the objectsurface, and the generation section generates, as the compression targetdata, the integrated data that includes the coordinate informationregarding each position of the object surface and the identificationinformation identifying the object existing at the position indicated bythe coordinate information. Therefore, the compression ratio can furtherbe increased. Additionally, the data acquisition section 101 mayacquire, for example, three-dimensional voxel data having height, width,and depth as well as two-dimensional image data having height and width.In such a case, segmentation conversion is performed on thethree-dimensional voxel data, and subsequent processing may be performedon the three-dimensional voxel data.

Further, in the present embodiment, the data storage section 104converts the compression target data to data obtained by normalizing andquantizing the coordinate information included in the compression targetdata, and then compresses the resulting converted data. This makes itpossible to increase the compression ratio and compress theidentification information as is. Consequently, the values representingthe identification information can be prevented from being changed byquantization.

Second Embodiment

FIG. 13 is a diagram illustrating an example configuration of the datastorage section 104 included in the information compression systemaccording to a second embodiment of the present disclosure. The datastorage section 104 depicted in FIG. 13 includes, as compressionprocessing components, an ID conversion compressor 301 and a voxelencoder 302 in addition to the components depicted in FIG. 12 , andincludes, as decompression processing components, an ID conversiondecompressor 311 in addition to the components depicted in FIG. 12 , anda voxel decoder/point clouder 213 a in place of the point clouder 213.

In the compression processing, the ID conversion compressor 301 replacesthe values representing the identification information according to theobject and the similarity of sense, which are indicated by theidentification information. The ID conversion compressor 301 is built,for example, by a learned model based on the use of the DNN.

FIGS. 14A and 14B are diagrams illustrating a conversion processperformed by the ID conversion compressor 301. As depicted in FIG. 14A,the “sense,” which is usually identified by the sense ID, is setindependently of the value of the sense ID. Therefore, the distancebetween sense ID values is independent of the similarity of “sense”(semantic distance). Consequently, if the sense ID is compressed as is,the “sense” after decompression might significantly be changed from the“sense” before compression due to a sense ID value shift by compression.

Accordingly, the ID conversion compressor 301 converts the sense IDvalues according to the sense as depicted in FIG. 14B. For example, theID conversion compressor 301 performs conversion in such a manner thatthe sense ID values close to each other in semantic distance, such as“vehicle” and “road,” approximate to each other.

FIGS. 14A and 14B use integers as the sense ID values before conversion.The sense ID values after conversion are not limited to integers.Further, the sense ID values after conversion may have a width. Forexample, in a case where the value of a sense ID is within a range of0.5 to 1.1, the sense ID may represent a “vehicle” as its sense.Further, although FIGS. 14A and 14B illustrate sense IDs, the IDconversion compressor 301 may additionally convert instance IDs in thesame manner as for the sense IDs.

Returning to FIG. 13 , the voxel encoder 302 quantizes the voxelinformation generated by the voxelizer 202 and converts the quantizedvoxel information to a feature value map by encoding the quantized voxelinformation through irreversible conversion. The voxel encoder 302 isbuilt by a learned model based on the use of the DNN such as a CNN.Further, the most frequent value representing the identificationinformation included within a range of such quantization is selected asa quantized voxel value.

The entropy estimator 203 estimates the probability distribution as theentropy of the feature value map generated by the voxel encoder 302. Theentropy encoder 204 generates the compressed binary data by encoding thefeature value map according to the estimated probability distribution.

In the decompression processing, the entropy decoder 211 generates thefeature value map by decoding the compressed binary data. The voxeldecoder/point clouder 213 a generates the voxel information by decodingthe feature value map generated by the entropy decoder 211, and convertsthe generated voxel information to the quantized chunk data thatincludes the coordinate information and the identification information.

The inverse quantizer/renormalizer 214 inverse-quantizes andrenormalizes the coordinate information regarding the quantized chunkdata. The ID conversion decompressor 311 generates the chunk data, whichserves as the decompressed data, by performing inverse conversion, whichis the inverse of conversion performed by the ID conversion compressor301, on the ID information regarding the quantized chunk data.

As described above, according to the present embodiment, theidentification information is compressed after the value representingthe identification information is replaced according to the similarityof sense. Therefore, even when the identification information isirreversibly compressed, it is possible to suppress a shift in thesense. Consequently, the compression ratio can be increased.

Third Embodiment

FIG. 15 is a diagram illustrating a configuration of the data storagesection 104 included in the information compression system according toa third embodiment of the present disclosure. The data storage section104 depicted in FIG. 15 includes a sense ID conversion compressor 401,an instance ID conversion compressor 402, and a point cloud compressor403 as compression processing components.

In the present embodiment, the data storage section 104 processes eachchunk data piece in the compression target data as list format data (x,y, z, S, I) that includes the coordinate information (x, y, z) and theidentification information (S, I) as components.

The sense ID conversion compressor 401 and the instance ID conversioncompressor 402 form an ID color converter that converts theidentification information (S, I) to color information formatinformation (R, G, B). Therefore, the list format data (x, y, z, S, I)is converted to list format data (x, y, z, R, G, B) that uses the colorinformation. The color information (R, G, B) included in the list formatdata (x, y, z, R, G, B) is obtained by converting the identificationinformation (S, I). Consequently, unlike the color information includedin the original color camera data, the color information (R, G, B)included in the list format data (x, y, z, R, G, B) can reducerandomness and increase the compression ratio.

The point cloud compressor 403 generates the compressed binary data thatis obtained by compressing the list format information (x, y, z, R, G,B) as point cloud data having color information. An existing compressorfor compressing the point cloud data may be used as the point cloudcompressor 403.

The data storage section 104 includes, as decompression processingcomponents, for example, a point cloud decompressor for generatingdecompressed data by decompressing the compressed binary data generatedby the point cloud compressor 403, and a color ID converter forgenerating the list format data (x, y, z, S, I) by inversely convertingthe identification information regarding the decompressed data generatedby the point cloud decompressor (neither of these decompressionprocessing components are depicted in FIG. 15 ).

Further, the above description assumes that the sensor data includes thepoint cloud data and the color camera data. However, the point clouddata need not always be included in the sensor data. In the case wherethe point cloud data is not included in the sensor data, for example,the data acquisition section 101 may acquire the point cloud data fromthe color camera data by analyzing the color camera data and estimatingthe positions of the object surface or may compress the sensor datawithout using the point cloud data.

FIG. 16 is a diagram illustrating an example configuration of the datastorage section 104 in a situation where the sensor data is to becompressed without using the point cloud data. The data storage section104 depicted in FIG. 16 includes a sense ID conversion compressor 411,an instance ID conversion compressor 412, and a video compressor 413 ascompression processing components.

In the example of FIG. 16 , the data storage section 104 regards eachchunk data piece in the compression target data as three-dimensionallyarrayed data ([2] [x] [y]) that represents the identificationinformation (S, I), the pixel position in the horizontal axis direction,and the pixel position in the vertical axis direction. Here, [2]represents the identification information, [x] represents the pixelposition in the horizontal axis direction, and [y] represents the pixelposition in the vertical axis direction.

The sense ID conversion compressor 411 and the instance ID conversioncompressor 412 form an ID color converter that converts theidentification information (S, I) to color information formatinformation (R, G, B). Therefore, the three-dimensionally arrayed data([2] [x] [y]) is converted to three-dimensionally arrayed data ([3] [x][y]) that uses the color information. The color information [3] includedin the three-dimensionally arrayed data ([3] [x] [y]) is obtained byconverting the identification information (S, I). Consequently, unlikethe color information included in the original color camera data, thecolor information [3] included in the three-dimensionally arrayed data([3] [x] [y]) can reduce the randomness and increase the compressionratio.

The video compressor 413 compresses information in a three-dimensionalarray ([3] [x] [y]) as image data (more specifically, video data). Anexisting compressor for compressing a video may be used as the videocompressor 413.

The data storage section 104 includes, as decompression processingcomponents, for example, a video decompressor for generatingdecompressed data by decompressing the compressed binary data generatedby the video compressor 413, and a color ID converter for generating thethree-dimensionally arrayed data ([2] [x] [y]) by inversely convertingthe identification information regarding the decompressed data generatedby the video decompressor (neither of these decompression processingcomponents are depicted in FIG. 16 ).

In the case where the configuration depicted in FIG. 16 is adopted, theprocessing by the integration section 103 is omitted. Further, in thecase where the configuration depicted in FIG. 16 is adopted, a spatialrange of data to be decompressed is designated, for example, byspecifying the sensor. Alternatively, the spatial range of the data tobe decompressed may be designated by specifying the position. In such acase, the sensor ID is determined from the specified position accordingto a sensor information table.

As described above, in the present embodiment, the compression targetdata is compressed after being converted to the list format data orthree-dimensionally arrayed data that includes the color information.Therefore, the efficiency of compression can be increased by using anexisting compressor. Further, the point cloud data can be acquired fromthe image data. This eliminates the necessity of using, for example, asensor for acquiring the point cloud data.

Fourth Embodiment

FIG. 17 is a diagram illustrating a configuration of the segmentationsection 102 and data storage section 104 included in the informationcompression system according to a fourth embodiment of the presentdisclosure. In the example of FIG. 17 , the sensor data does not includethe point cloud data. Further, the segmentation section 102 and the datastorage section 104 are integrally formed.

The segmentation section 102 and the data storage section 104 include anencoder 501, an entropy estimator 502, and a decoder 503 as compressionprocessing components, and include a generative decoder 504 as adecompression processing component.

The encoder 501 converts an input image (image[3] [x] [y]), namely, thecolor camera data, to encoded data (z[c] [a] [b]) by encoding andcompressing the input image through irreversible conversion. The encodeddata is, for example, the feature value map.

The entropy estimator 502 estimates the entropy (probabilitydistribution) of the encoded data generated by the encoder 501, andperforms an entropy encoding process and an entropy decoding process ina manner similar to FIG. 12 .

The decoder 503 generates output segmentation data (seg[2] [x] [y]),namely, the compressed binary data, by decoding the encoded datagenerated by the encoder 501.

The encoder 501 and the decoder 503 are built by a learned model basedon the use of the DNN such as the CNN. For building a learned model bymachine learning, for example, end-to-end learning of both thesegmentation section 102 and the data storage section 104 is used. As aloss function for learning, for example, “Loss=λ*entropy+distortion (segdata, training seg data)” is used. Here, λ is a parameter fordetermining rate-distortion trade-off, and “entropy” is the entropy (theamount of information) calculated by the entropy estimator 502. Further,the seg data is the output segmentation data, and the training seg datais training data for the output segmentation data. A distortion functionis, for example, typically a general loss function for segmentation suchas cross entropy or a mean squared error (MSE). Alternatively, forexample, a differentiable image quality index regarding images, such asa multi-scale structural similarity (MS-SSIM) index, may be used.

The generative decoder 504 generates output image data (image[3] [x][y]), which includes color information as the decompressed data, bydecoding the output segmentation data. The generative decoder 504 isbuilt, for example, by a DNN-based model that is learned by a generativeadversarial network (GAN). The generative decoder 504 may be built bylearning that is different from learning for generating the encoder 501and the decoder 503.

As described above, according to the present embodiment, thesegmentation section 102 and the data storage section 104 are integrallybuilt by a learned model. Therefore, the configuration of theinformation compression system can be simplified. Further, as thepresent embodiment outputs the image data to which the color informationis attached by decompression processing, for example, data analysis canbe made by using an application program that is in the same format asconventional application programs.

Fifth Embodiment

FIG. 18 is a diagram illustrating a configuration of the data storagesection 104 included in the information compression system according toa fifth embodiment of the present disclosure. The data storage section104 depicted in FIG. 18 includes a normalizer 601, a point cloud encoder602, an entropy estimator 603, and an entropy encoder 604 as compressionprocessing components, and includes an entropy decoder 611, an entropyestimator 612, a voxel decoder 613, and a renormalization/meshgeneration section 614 as decompression processing components.

In the present embodiment, the chunk data is configured on the basis ofindividual pieces of identification information for the purpose ofcompressing the coordinate information. Therefore, the identificationinformation is retained as one of items of the chunk information 60.

The normalizer 601 normalizes the coordinate information. The pointcloud encoder 602 generates the feature value map by encoding andquantizing the normalized coordinate information. The feature value mapis a data array that has a predetermined size and values converted tointegers by quantization. The point cloud encoder 602 is built, forexample, by a DNN having a combination of a multilayer perceptron (MLP)and a max-pooling layer.

The entropy estimator 603 estimates the probability distribution as theentropy of the feature value map generated by the point cloud encoder602. The entropy estimator 603 is built, for example, by a learned modelbased on the use of the DNN such as a multilayer one-dimensionalconvolutional neural network.

The entropy encoder 604 generates the compressed binary data by encodingthe feature value map according to the probability distributionestimated by the entropy estimator 603.

Further, in the decompression processing, the entropy decoder 611generates the feature value map by decoding the compressed binary data.Specifically, the entropy decoder 611 generates the feature value map byusing the entropy estimator 612. More specifically, the entropy decoder611 predicts the probability distribution of values (symbols) in thefeature value map by using the entropy estimator 612, and the entropydecoder 611 uses the compressed binary data and the predictedprobability distribution to decode the symbols, and eventually performsdecoding to generate the feature value map.

The voxel decoder 613 decodes the feature value map, which is generatedby the entropy decoder 611, on the basis of individual three-dimensionalregions designated as the voxels, and generates occupancy informationindicating occupancy of an imaging target object in the individualregions designated as the voxels. Unlike the point cloud, whichgenerally contains the position information regarding only the surfaceof an imaged object, the value of occupancy is close to 1 in a regioninside the imaged object and close to 0 in a region outside the imagedobject. Using voxelized occupancy data makes it possible to obtain amesh close to an original object. By utilizing such characteristics andadditionally using entropy encoding, it is possible to accurately storeinformation regarding the imaged object with a smaller amount of data.The voxel decoder 613 is built, for example, by a learned model based onthe use of the DNN having the MLP.

The renormalization/mesh generation section 614 generates the coordinateinformation and surface information by renormalizing and meshing theoccupancy information. The surface information indicates a surface thatis expressed by a set of three or more points indicated by thecoordinate information. Meshing is performed by using, for example, amarching cubes method.

As described above, according to the present embodiment, the coordinateinformation is compressed on the basis of individual pieces ofidentification information. Further, according to the presentembodiment, mesh information can be obtained by decoding the compressedbinary data. Therefore, the mesh data can efficiently be read out fromthe data usage section 106 without waste and without going through thedata conversion section 105.

The foregoing embodiments of the present disclosure are illustrative andnot restrictive of the present disclosure. It is to be understood thatthe scope of the present disclosure is not limited to the foregoingembodiments. A person skilled in the art is able to carry out thepresent disclosure in various different modes without departing from thescope of the present disclosure.

What is claimed is:
 1. An information compression system that compressesdata, the information compression system comprising: an acquisitionsection that acquires the data; a generation section that determineseach object depicted by the data and a sense of each object, andgenerates compression target data according to results of thedetermination, the compression target data being obtained by convertingvalues of individual elements in the data to identification informationthat indicates each object and the sense of each object; and acompression section that generates compressed data by compressing thecompression target data.
 2. The information compression system accordingto claim 1, wherein the acquisition section further acquires point clouddata that includes a plurality of pieces of coordinate informationindicating individual positions of a surface of the object, and thegeneration section generates integrated data, as the compression targetdata, according to the point cloud data and the results of thedetermination, the integrated data including the coordinate informationregarding each position of the surface of the object and theidentification information identifying the object existing at theposition indicated by the coordinate information.
 3. The informationcompression system according to claim 2, wherein the acquisition sectionacquires the point cloud data from the data.
 4. The informationcompression system according to claim 2, wherein the compression sectionconverts the compression target data to data that is obtained bynormalizing and quantizing the coordinate information included in thecompression target data, and compresses the resulting converted data. 5.The information compression system according to claim 2, wherein thecompression section converts the compression target data to data that isobtained by replacing values representing individual pieces ofidentification information included in the compression target dataaccording to the object and similarity of the sense, and compresses theresulting converted data.
 6. The information compression systemaccording to claim 2, wherein the compression section converts thecompression target data to list format data that is obtained byarranging values representing the coordinate information andcolor-indicating values generated from individual pieces ofidentification information included in the compression target data, andcompresses the resulting converted data.
 7. The information compressionsystem according to claim 1, wherein the compression section convertsthe compression target data to three-dimensionally arrayed data that hasvalues of the identification information as values of individualelements, and compresses the resulting converted data.
 8. Theinformation compression system according to claim 1, wherein thegeneration section and the compression section are integrally built by alearned model.
 9. The information compression system according to claim2, wherein the compression section compresses coordinate informationincluded in the compression target data on the basis of the individualpieces of identification information.
 10. The information compressionsystem according to claim 1, further comprising: a decompression sectionthat generates decompressed data by decompressing the compressed data;and a generation section that processes the decompressed data andoutputs the resulting processed data.
 11. The information compressionsystem according to claim 10, further comprising: a data usage sectionthat provides an interface for designating at least either a space whereor a time when the data is acquired, wherein the decompression sectiondecompresses the compressed data according to the designation made bythe interface.
 12. An information compression method that is used by aninformation compression system for compressing data, the informationcompression method comprising: acquiring the data; determining eachobject depicted by the data and a sense of each object, and generatingcompression target data according to results of the determination, thecompression target data being obtained by converting values ofindividual elements in the data to identification information indicatingeach object and the sense of each object; and generating compressed databy compressing the compression target data.